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Abstract 

We consider the invariant measure of a homogeneous continuous- 
time Markov process in the quarter-plane. The basic solutions of the 
global balance equation are the geometric distributions. We first show 
that the invariant measure can not be a finite linear combination of 
basic geometric distributions, unless it consists of a single basic geo- 
metric distribution. Second, we show that a countable linear combina- 
tion of geometric terms can be an invariant measure only if it consists 
of pairwise-coupled terms. As a consequence, we obtain a complete 
characterization of all countable linear combinations of geometric dis- 
tributions that may yield an invariant measure for a homogeneous 
continuous-time Markov process in the quarter-plane. 
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1 Introduction 



Homogeneous continuous-time Markov processes in the quarter-plane have 
translation invariant transition rates, except for the rates along the horizontal 
and vertical boundaries. In literature, many examples exist of such processes 
with geometric invariant measure, including Jackson networks and queuing 
networks with negative customers, see j4j for an overview. The compensation 
approach of Adan [2j |lj has revealed that Markov processes without transi- 
tions to the east, north and northeast can have an invariant measure that is 
a countable linear combination of geometric terms. 

The present paper provides a complete characterization of all countable 
linear combinations of geometric distributions that may yield an invariant 
measure for homogeneous Markov processes in the quarter-plane. In par- 
ticular, our contributions are as follows. First we show that the invariant 
measure cannot be a finite linear combination of geometric distributions, 
unless it consists of a single geometric distribution. Second, we show that a 
countable linear combination of geometric terms can be an invariant measure 
only if it consists of pairwise-coupled terms, i.e., only if each 2-dimensional 
geometric distribution in the sequence shares a parameter with the previous 
term in the sequence. 

Alternative approaches to analyzing the invariant measure of Markov pro- 
cesses in the quarter-plane are available in the literature. Most notably, gen- 
erating functions have been used for the analysis of a variety of such problems, 
see, e.g., [TJ |9]. A general theory is provided in [6l [8], in which the invariant 
measure is characterized via transforms. However, an explicit expression for 
the invariant measure is usually hard to obtain. The present paper char- 
acterizes invariant measures of a tractable form, i.e., linear combinations of 
geometric terms. 

The remainder of this paper is structured as follows. In Section |2] we 
present the model and some definitions. The main results of this paper are 
given in Section |3l 
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2 Model and Definitions 



2.1 Model 

Consider a two-dimensional continuous-time Markov process on the pairs 
of non-negative integers. We refer to > 0,j > 0}, > 

0,j = 0}, = 0,j > 0} and (0,0) as the interior, the horizontal 

axis, the vertical axis and the origin of the state space, respectively. The 
transition rate from (i, j) to (z + s,j +t) is denoted by qs,t{i,j)- The process 
is homogeneous in the sense that for any and [k, I) in the interior of the 
state space 

qs,t{i,j) = qsA^^J) and qs,t{i - s, j - t) = qs,t{k - s,l - t). (1) 

for all s and t. Moreover, holds for all pairs {k, I) on the horizontal 

and vertical axis respectively. Note that the first equality of ([1]) implies 
that the rates of transitions leaving from each part of the state space are 
translation invariant. The second equality ensures that also rates entering 
the same part of the state space are translation invariant Q Transitions are 
restricted to adjoining points (horizontally, vertically and diagonally), i.e., 
qs,t{k,l) = if |s| > 1 or |t| > 1. We introduce, for i > 0, j > 0, the 
notation qs,t{i,j) = qs,t, qs,o{i,0) = hs and go,i(0,j) = f*. Finally, let -go,o, 
—ho and —vq denote the outgoing rates in the interior, at the horizontal axis 
and at the vertical axis respectively. The model and notation are illustrated 
in Figure [H We will refer to this type of process as a homogeneous Markov 
process. In the remainder of this paper, if not explicitly stated, we assume 
that a Markov process is ergodic. 



2.2 Candidate geometric measures 

Our interest is homogeneous Markov processes with an finite invariant mea- 
sure that can be expressed as a linear combination of geometric measures. 
In particular, we will consider geometric measures that satisfy the balance 
equations in the interior of the space, i.e., measures of the form pV-^, with 

"'^Note that this is a stronger notion of homogeneity than considered in, for instance, [I]. 
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Figure 1: Homogeneous Markov process. 



(p,(t) G C, 



1 1 



-1 t=-l 



(2) 



If m is a linear combination of terms F C C we say that m is induced by 



F, see Figure 2 (a 



Definition 1 (Induced measure). Measure m is called induced by T G C if 



(p,<^)er 

with a{p, 0") > for all (p, a) G F. 

We assume that m is finite. In this case, o-)Gr "^(P' '^)(-'- ~ ~ 
(t)~^ < oo. We restrict our attention to F of finite and countably infinite 
cardinality. 

2.3 Uncoupled partitions 



Different ways of partitioning set F will be introduced here. These partitions 
play an essential role in the analysis later on. 
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Figure 2: Partitions of F. In (a) an example of curve C and subset F are 
given. Figures (b) , (c) and (d) give the uncoupled, horizontally uncoupled 



and vertically uncoupled partitions of F respectively, where different compo- 
nents are marked by different symbols. 



Definition 2 (Uncoupled partition). A partition {Fi,F2,---} of T is 1) 
horizontally uncoupled if {p,cr) e Fj and (p, a) G Tj for i ^ j, implies that 
p 7^ p; is 2) vertically uncoupled if {p,<j) G Fj and (p, ct) G Fj for i ^ j, 
implies that a ^ a; and is 3) uncoupled if it is both horizontally and vertically 
uncoupled. 

We call the partition with the largest number of components a maximal 
partition. 

Lemma 1. Among all the horizontally uncoupled partitions, there exists a 
unique maximal horizontally uncoupled partition. 

Proof. Assume that {Ti}^^ and {F-}^]^ are different maximal horizontally 
uncoupled partitions of F. W.l.o.g. Fi fl F'^ 7^ and Fi \ F'^ 7^ 0. Consider 
(p, cr) G Fi \F'^ and (p, a) G Fi flF'^ If p = p, then {F-}j is not a horizontally 
uncoupled partition. If p 7^ p, then {Fj}j is not maximal. □ 

Existence of maximal unique (vertically) uncoupled partitions follows sim- 
ilarly. Examples of maximal uncoupled partition, horizontally, vertically un- 



coupled partitions can be found in Figures 2(b), 2(c) and 2(d) respectively. 
We denote the number of components in the maximal horizontally uncou- 
pled partition by H and the components themselves as F^, i = 1, - ■ ■ ,H. 
The common horizontal coordinate of set F^ is denoted by ^?(F^). The max- 
imal vertically uncoupled partition has V components, FJ, j = I,-- - ,V, 
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where elements of have common vertical coordinate ^(Fp.The maximal 
uncoupled partition is denoted by {F^}^^^. The elements of this partition 
can be obtained by taking the union of elements from {Ff}^^ or {FJ}]^^. 
For k = 1, . . . ,U, let Ik C {1, . . . , H} and Jk C {1, . . . ,V} be such that 
F^ = Uie/fe ~ UjeJfc Using the maximal uncoupled partition, we can 
introduce measures rrik, defined as 

mkiij)= 5Z "(P'O-)pV^. (3) 

(p,'^)er« 

This allows us to write m{i,j) = Ylk=i ^k{hj)- 

Remark: Note that H, V and U are not necessarily finite. With a slight 
abuse of notation we will write expressions involving for instance Ylk=iy 
k = 1,...,H, also when H is infinite, i.e., the number of components is 
countably infinite. The same assumption holds for V. 

3 Analysis 

We consider the structure and cardinality of F. In Section [3TT] we consider the 
number of uncoupled components in F. In Section [X^ we study the structure 
of F in more detail. Finally, in Section [X^ we consider the cardinality of F. 

3.1 Number of uncoupled components in F 

The following theorem is the first main result. It states that an invariant 
measure can not be induced by a set F of which the uncoupled partition 
contains multiple components. 

Theorem 1. Consider a homogeneous Markov process P and its invariant 
measure m. If m is induced by T G C , then U = 1, i.e., the maximal 
uncoupled partition {F", . . . , T^} ofT consists of a single component. 

The proof of the theorem is deferred to the end of this section. We first 
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introduce some additional notation. For any set Fj C F let 



{p,'^)eri 



s=-l 

1 



B^{T,)= «(P,^)E (^'"*^* + /^^'"Vl,^)• 



(4) 
(5) 



t=-i 



Note that B^{r) and B^[r) are the balance equations for the measure induced 
by F at the horizontal and vertical boundary respectively. 

Lemma 2. Consider a homogeneous Markov process P and any finite mea- 
sure m induced by some F C C. The sequences {B'^iV'^)}^^^ and {i?*'(F^)}^]^ 
are absolutely convergent. 

Proof. W.l.o.g. we will only prove that the sequence {B^{V^)}^^^ is absolutely 
convergent. 



H 



H 



i=l 
H 



^ a(p,a) ^ (p^ 'K + p^ '(Tqs,-i) 



s=-l 



s=-l 



(p,^)6r 



p 1 — cr 



< oo, 



where the last equality follows from the fact that m is a finite measure, see 
also Definition [H □ 

Note that Lemma [2] does not require m to be the invariant measure of P, 
it can be any finite measure. The following lemma is a key element for the 
proof of Theorem [H 

Lemma 3. Consider a measure m and homogeneous Markov process P. Let 
m be induced by T G C. Then m is the invariant measure of P if and only 
if for alll<i<H,l<j<V, 5^(Ff) = and B''{T]) = 0. 
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Proof. Since m is the invariant measure of P, it satisfies the balance equations 
at state {i,0). Therefore, 



1 



= [m{i — k, 0)/ifc + rn{i — k, 1)^^ _i] 



k=-l 

1 



(6) 



(p,<T)Gr k=-i 



H 



1 



1 



(7) 



The exchange of summations is justified by Lemma [21 

Suppose that H is finite. From ^ it follows that ^''(r,^), l<i<H, sat- 
isfy a Vandermonde type system of equations. Moreover, from the properties 
of a maximal horizontally uncoupled partition, the coefficients q{T^) are all 
distinct. It follows that B^{T^) = 0, 1 < i < H . For countably infinite H we 
resort to [51 Theorem 1], which can be applied based on Lemma [21 Using the 
same reasoning it follows that B'"{r'^) = 0, 1 < z < l^, finishing one direction 
of the proof. 

Validity of the other direction can be readilly verified by observing that, if 
B^{T^) = 0, then ^J^^ ^''(r,'^) = and the balance equation for (i, 0), i > 
is satisfied. Using the same reasoning balance at the vertical axis is satisfied. 
Balance in the interior is satisfied by the assumption that m is induced by 
r C C. Finally, balance in the origin is implied by balance in other parts of 
the state space. □ 

Proof of TheoremUl We will show that the measures m^, k = 1, . . . ,U, sat- 
isfy all balance equations. Let measure rrik be induced by Ffc. By definition 
of C this implies that all m^, k = 1, . . . ,U satisfy the balance equations in 
the interior of the state space. For the balance equation for at state {i, 0) 
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at the horizontal boundary we obtain 
1 

[mk{i - s, 0)hs + mk{i - s, -i] 

s=-l 

1 

1 

{p,a)eri s=-i 

1 

= 0. 

By Lemma [21 the interchange of the summations leading to the second equal- 
ity is valid. The last equality follows from Lemma [31 

In similar fashion it follows that the balance equations at the vertical 
boundary are satisfied. As a consequence, we have shown that mi, ■ ■ ■ ,mu 
are invariant measures of P. When U > 1, this contradicts to the fact that 
there is a unique invariant measure for ergodic Markov process P. □ 

3.2 Structure of T 

In this section the structure of set F will be discussed. From Theorem [H 
it follows if the number of components in the maximal uncoupled partition 
is greater than one, then a measure induced by F cannot be the invariant 
measure of homogeneous Markov process P. In this section we investigate 
the measure induced by a set with one uncoupled component. To this end, 
we introduce the notion of a pairwise-coupled set. 

Definition 3 (Pairwise-coupled). A countable ordered subset F o/ C, F = 
{{Pk, CTk), k = 1, 2, 3 ■ ■ ■ } is a pairwise-coupled set if and only if one of the 
following is true. 

1) pi = P2, O"! > 0-2, P2 > P3, 0-2 = (Xs, ps = P4, as > (74, ■ ■ ■ , 
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2) pi > P2, (^1 = 0^2, P2 = P3, > CTS, P3 > Pi, (^3 = (^4, ' ' ' , 

3) Pi < P2, Ol = 0-2, P2 = P3, 0-2 > CTS, P3 < P4, = 0-4, " " " , 
-^y* Pi = P2, CTl > 0-2, P2 < Ps, 0-2 = O-3, P3 = P4, (T3 > (74, ■ ■ ■ . 

Pairwise coupling allows the explicit characterization of the structure of 
set r that is required if the measures induced by F are the invariant mea- 
sure of homogeneous Markov process P. The following corollary is a simple 
application of Theorem [H 

corollary 1. Consider a homogeneous Markov process P and its invariant 
measure m. If m is induced by T G C, then T is a pairwise- coupled set. 

3.3 Cardinality of T 

From Sections 13. II and [ 3.21 we know that F consists of a single component and 
is hence pairwise-coupled. The next theorem characterizes the cardinality of 
this component. 

Theorem 2. Consider a homogeneous Markov process P and its invariant 
measure m. If m is induced by V d C , then F can contain either one or 
countably many elements. 

The proof of this theorem follows from Lemma H] and [7l that deal with 
the cases of |F| = 2 and 2 < |F| < oo, respectively. 

Lemma 4. Consider a homogeneous Markov process P and its invariant 
measure m. If m is induced by a pairwise-coupled set F C C, then |F| ^ 2. 

Proof. Suppose that 

'm{i,i) = a{p,a)p'a^ + a{p,a)p'a\ (8) 

where (p, a) E C and (p, a) G C. 

It follows from the definition of C that a and a are the roots of the 
following quadratic equation in x, 

1 1 

E P'^^ls^kx'-" = 0. (9) 

fc=-l s=-l 

Note that the maximal vertically uncoupled partition of {(p, a), (p, 5")} 
consists of the two singleton components {(p, a)} and {(p, ct)}. It follows 
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from Lemma [2] that B^{{{p, a)}) = B^{{{p, a)}) = 0. Therefore, a and a are 
the roots of 

1 

J2{PQ-i,s + Vs)x'-' = 0. (10) 

s=-l 

From a comparison of the coefficients of ([9j) and (fTOl) it follows that either 
a) one of the roots will be 1, contradicting the definition of C, or b) the 
transition rates of P are such that P is not irreducible and hence not ergodic. 
Hence, m as defined in ([8]) can not be the invariant measure of P. Using the 
same argument it follows that a form a{p, a)p^a^ + ^(Pj o')p^a^ cannot be the 
invariant measure of P. □ 

Before proving the final lemma, i.e., that F satisfying 2 < |F| < oo can 
not induce an invariant measure, we introduce a final piece of notation and 
some technical results that will help in the presentation of the remaining 
proofs. Observe that B^{T'l) = is a linear relation in hi and Let 
6^(Ff) be defined as 

B'^iTl) = ^ b\Tl) = (l - h + {l- giV^)) h.i. (11) 

Analogously we define IfiTI) as 



B\T\) = ^ b\r,) = (^1 - vi + il- v_i. (12) 

The following technical result will greatly simplify the presentation of our 
final proofs of the cases when the measure is induced by F satisfies 2 < |F| < 
oo. 

Lemma 5. If a > a and p > p then 

b\{{p, a), (p, a)}) > b\{{p, a)}), b\{{p, a), (p, a)}) < b\{{p, a)}), 
b^{{{p, a), (p, a)}) > b\{{p, a)}), 6^({(p, a), (p, a)}) < b^ip, a)}). 

Proof. From the definition in (fTT!) it follows that 

,ft/r/ N / a(p,a)a + a(p,a)d- , 1 , 

b\{{p, a), p, a)}) = 7 \ pg_i,,i + go.-i + - 

a(p, cr) + a(p, a) p 
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and 



b''{{{p, a)}) = a(pg_i _i + go -i + -i) ~ ^i-i ~ ^0,1 - ^-1,1, 



b^iiiP^ cr)}) = (^{pQ-i-i + %-! + -Qi-i) - 11,1 - %,i - (1-1,1- 

P 



From the above the first row of inequahties follow directly. The remaining 
inequalities follow directly from ( !T2|) . □ 

The second technical lemma that we will need is readily verified and 
stated without proof. 

Lemma 6. - p) + t2(l - p) > 0, - 1/p) +^2(1 - 1/p) > and 
p > p, then ti < and t2> 0. 

The final result, together with LemmaHlit provides the proof of Theorem[2] 
is the following lemma. 

Lemma 7. Consider a homogeneous Markov process P, if 2 < |r| < 00, 
then no measure induced by T can be in the invariant measure of P. 

Proof of Lemma Now we need to show the the measure of the form m{i, j) = 
Ym=i pI'^i with n < 00 can not be an invariant measure. W.l.o.g. we can as- 
sume that pi = P2, cTi > 0"2 and cr2 > as > ■ ■ ■ > cr„. Moreover, ?(rj) is 
strictly decreasing with j. We deal with two cases separately. The first case 
is P2 > ps, ps > P4 > ■ ■ ■ > Pn and Q{Tf) is strictly decreasing with i. The 
second case is p2 < pa, pa < pa < • • ■ < Pn and giVf ) is strictly increasing 
with i. 

For the first case we consider the relations 

(i-i/pi)/ii + (i-Pi)/i_i = 6'^(r?), 
(i-i/p„)/^i + (i-p„)/.„i = 6'^(r^), 

(1 - IK) + (1 - ^nKi = &"(r^), 

which by Lemma [3] are required to hold if m is the invariant measure of P. 
We will construct Si, $2, ti and ^2 that satisfy 

(l-l/pi)si + (l-l/p„)s2 >0, 

(l-pi)si + (l-p„)s2 >0, 
(l-l/ai)ti + (l-l/a„)t2 >0, ^ ' 

(l-(Ti)ti + (l-a„)t2 >0 
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and 

6^(r^)si + b''iT%)s2 + fe"(r^)ti + b''iry)t2 < o. (15) 

By Farkas' Lemma this leads to a contradiction to f|T3l) . 

The si, S2, ti and ^2 are constructed by considering the auxihary measure 
ffi = a{pi, ai)p\a{ + Q;(p„, an)PnCri and the homogeneous Markov process P, 
that has the same transition rates as P in the interior of the state space 
and rates hi, h_i, vi and along the boundaries. We now consider the 
rclcXt ions 

(1 - l/pi) h + pi)h_i = b'^iiipi, ai)}), 

(1 - 1/pn) h + il- p„)/^-l = b\{ipn, an)}), 

(l-lM)t;i + (l-ai)t;_i=6''({(pi,ai)}), ^ ^ 

(1 - l/crn)vi + (1 -an)v^i = 6''({(p„,Cr„)}). 

If (ITB]) would hold for hi = hi, h_i = h_i, Vi = Vi and v^i = v_i, fh would 
be the invariant measure of P which contradicts the assumption that m is 
the invariant measure of P. However, if P is ergodic f|T6|) can not hold due to 
Theorem [TJ If P is not ergodic, there is no finite invariant measure and f|T6l) 
can not hold either. Therefore, f|T6l) is not satisfied for any non-negative 
hi, h^i, Vi and v^i. By Farkas' Lemma, there exist Si, S2, ti and ^2 that 
satisfy ffT^ and 

b\{{pi, (yi)})si + 6''({(p„, a^)})s2 + W{{{pi, ai)})ti + 6-({(p„, 0^)})^ < 0. 

Note, that from Lemma[5]it follows that h^{{T^l}) < 6^({(pi, ai)}), ^''({r^}) > 
h\{{Pn,an)}), h\{V\}) = b^i{ipi,ai)}) and b^iV^y}) > ^^{(Pn, a^}). 
Also, from Lemma [H] it follows that si > 0, S2 < 0, ti > 0, t2 < 0. Therefore, 
Si, S2, ti and t2 satisfy ffT^ . This concludes the proof of the first case. 
For the second case we consider the relations 

{l-l/ai)vi + {l-ai)v.i = b^{ri), 

{l-l/a2)vi + {l-a2)v^i = b%r,), ^ ^ 

that are necessary for m to be the invariant measure and obtain a contradic- 
tion by constructing ti and t2 that satisfy 

(l-l/ai)ti + (l-l/a2)t2 >0, (18) 
(l-ai)ti + (l-fT2)t2 >0, (19) 
6"(r^)ti + 6''(r^)t2<0. (20) 
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The auxiliary measure that is used is m{i,j) = a{pi, (Ti)p\al + a{p2, cr2)P2'^2- 
Observe that pi = p2 and that the corresponding relations are 

(1 - /ii + (1 - = b\{{p,, a,), (p2, ^2)}), 

(1 - v, + {l- a,)v^^ = 6^({(pi, a^)}), 

(1 - IM) vi + il- a2)v^i = 6"({(P2, ^2)}). 

From Farkas' Lemma and Lemma [Hit follows that there exist Si, ti and t2 
that satisfy (fTSj) . (fTOD and 

6'^({(pi, ai), (p2, a2)})s, + 6''({(pi, ai)})ti + 6''({(p2, a2)})t2 < 0, (21) 

where si = 0, since it satisfies (1 — l/pi)si > and (1 — pi)si > 0. More- 
over, since, b"(Tl) = 6''({(pi, (Xi)}) and, by Lemma [5l we have b^(Tl) > 
6^({(p2, (T2)}). Moreover, by Lemma Owe have, ti > 0,^2 < 0. Then it fol- 
lows that ti,t2 satisfy (I2U]) . This concludes the proof of the second case. □ 

To summarize the contributions of the present paper, we combine Theo- 
rems [U and [2] into the following corollary. 

corollary 2. Consider a homogeneous Markov process in the quarter-plane 
and its invariant measure m. If m is of the form 

(p,a)Gr 

then r is pairwise- coupled and has either one element or countably many. 

Note that if F has one element, then m as geometric product form and 
many examples exist in literature, see also [3]. The existence of invariant 
measures with countably many terms has been demonstrated in [2]. 
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